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ABSTRACT 

The Chapter 1 Developer/Demonstration Program, 
Learning to Read Through the Arts (LTRTA) , offers intensive reading 
and reading-oriented arts instruction to Chapter 1-eligible students. 
During the 1988-89 school year, the program served 100 ungraded 
special education students and 716 general education and limited 
English proficient (LEP) students in grades 2 through 6. LTRTA 
reading and artist teachers and participating classroom teachers work 
as a team to improve students* reading and writing skills and to 
increase their interest in the arts. Evaluation of the program was 
based on data from interviews of program staff, observation of 
program sites and classes, and analyses of standardized and 
criterion-referenced reading tests and holistically scored writing 
tests. Results indicated that (1) the progratm was implemented as 
proposed and was well organized and effective; (2) general education 
students in all grades except two made statistically significant mean 
gains on standardized reading tests; (3) 81% of the special education 
students mastered 3 or more additional reading skills, and 82% of the 
LEP students mastered 5 or more additional skills; (4) all students 
achiev3d statistically significant gains from pretest to posttest 
writing evaluations; and (5) the mean gain in reading achievement for 
1988-89 was more than twi'je as high as the mean gains for the 
previous two school year.^o Recommendations include monitoring the 
transportation problem and investigating the high turnover rate of 
artist teachers. (Nine tables of data are included.) (RS) 
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CHAPTER 1 DEVELOPER/DEMONSTIU^TOR PROGRAM: 
LEARNING TO READ THROUGH THE ARTS, 1988-89 
EVALUATION SUMMARY 

BACKGROUND 

The Chapter 1 Developer/Demonstration Program, Learning to 
Read Through the Arts (L.T.R.T.A. ) r offers intensive reading and 
reading-oriented arts instruction to Chapter 1-eligible students. 
During the 1988-89 school year, the program served lOO ungraded 
special education students and 716 students In grades two through 
six; 83 students were Limited English Proficient students, and 633 
were in general education classes. In 1988-89, the program was 
funded through Chapter 1 of the Education Consolidation and 
Improvement Act (E.C.I. A.) at $997,000. 

L.T.R.T.A. reading and artist teachers and participating 
classroom teachers work as a team to improve students' reading and 
writing skills and to increase their interest in the artis. Reading 
teachers provide an individualized, diagnostic, prescriptive 
approach to reading. Artist teachers stress listening, speaking, 
reading, and writing in the arts-oriented workshops, j^s students 
see, hear, touch, smell, and/ or taste their artistic creations, 
they perceive the artistic process and the product and recall what 
they have learned. The experience of learning to read through the 
arts helps build their self-confidence and improve their self- 
image. 

Program activities were based at four sites, one each in 
Manhattan, the Bronx, Brooklyn, and Queens. At each program site, 
there were two reading teachers. At the Manhattan site, there were 
three artist teachers, and at the Bronx, Queens and Brooklyn sites, 
there were two artist teachers and one office aide. The program 
director, two staff developers, and one office aide worked out of 
the program office, housed at P.S. 9 In Manhattan* 

PROGRAM OBJECTIVES 

The objectives for the 1988-89 L.T.R.T.A. program were: 

• General education students were expected to make 
statistically significant mean gains on standardized 
reading tests. 

• Sixty percent of the LEP students were expected to master 
at least five additional reading skills on the posttest 
of the Wisconsin Design Skills Development Test that they 
had not mastered on the pretest. 

• Sixty percent of the special nation students were 
expected to master at least t ee additional reading 
skills on the posttest of the Wisconsin Design Skills 
Development Test that they had not mastered on the 
pretest . 



• All students were expected to achieve statistically 
significant mean gains from pretest to posttest on 
holistically scored writing samples. 

The findings of this evaluation are based on data from 
interviews c program staff, observations of program sites and 
classes, and analyses of standardized and criterion-referenced 
reading tents and holistically scored writing tests. The impact 
of the program on student achievement in reading and writing was 
determined by evaluating students' performance on reading and 
writing tests against the program objectives. 

FINDINGS 

Site Observations and Interviews 

The review of program documents, observation of program 
activities, and interviews of program staff indicated that the 
program was implemented as proposed. The program was well 
organized and effective. Program staff were professional and 
collegial. Students were engaged and enthusiastic. However, sta : 
interviews revealed two problems. 

Transportation by bus was an occasional problem at some sites. 
Since all students must travel from their home school to the 
program sites and on field trips to cultural institutions in New 
York City, adequate transportation arrangements are important to 
the program's success. 

In general, artist teachers are not annualized — that is, not 
on the official Board of Education payroll as teachers. This is 
a source of anxiety to artist teachers. Program teachers believe 
it leads to a higher turnover rate among artist teachers than among 
reading teachers who are annualized and thus tc less program 
stability and continuity, both of which are essential to the team 
teaching concept practiced in the program. 

Reading Tests of General Educ ation Students 

General education students in all grades except grade two made 
statistically significant mean gains on standardized reading tests 
and thus met the program objectives. For students in fifth and 
sixth grade, these gains represented large effect sizes and thus 
educationally meaningful gains. The small decrease for grade two 
was neither statistically significant nor educationally neaningful. 
Since there were only 20 second grade students with complete test 
information, this finding should not be interpreted as a valid 
assessment of the program impact on this grade. 



Reading Te sts of L EP and Spe cial Education Students 



Eighty-one percent of special education students mastered 
three or more additional reading skills, and 82 percent of LEP 
students mastered five or more additional reading skills. Thus, 
the program surpassed its objectives. In addition, thci breakdown 
of LEP student reading achievement by grade showed that the program 
surpassed its objective for each grade. 

writing Tests 

Special education students and LEP and general education 
students in all grades achieved statistically significant mean 
gains from pretest to posttest and thus met the program objectives. 
The mean gains represented educationally meaningful gains. 

Comparii^on with Past Years 

Since the 1986-87 school year, the reading achievement of 
general education students has been measured with the Degrees of 
Reading Power test. For four years in a row, overall mean gains 
have been statistically significant. The mean gain of 9.9 N.C.E.s 
in 1988-89 is more than twice as high as the mean gains for the 
previous two school years. 

RECOMMENDATIONS 

Based on the evaluation findings and other information 
presented in the report, the following recommendations are made: 

• The program should continue its innovative curriculum of 
integrating reading and the arts and, as several teachers 
requested in the interviews, should expand, if possible, 
to serve an even larger population of students. 

• Given its importance to the program, transportation 
should be closely monitored by program staff so that any 
decline in service can be quickly identified and 
corrected . 

• The evaluation of the 1989-90 program should investigate 
whether or not the perceived higher turnover rate of 
artist teachers is a fact. If the turnover rate is 
substantially higher than that of annualized teachers, 
this could contribute to a lack of program stability and 
continuity. OREA, program staff, other Board of 
Education staff, and/or interested parties might then 
develop a proposal to change the status, of artist 
teachers in a feasible manner. 
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I . INTRODUCTION 

PROGRAM PURPOSE 

The Chapter 1 Developer/Demonstration Program, Learning to 
Read Through the Arts (L.T.R.T.A. ) / offers intensive reading and 
reading -oriented arts instruction to Chapter 1-eligible students. 
As students see, hear, touch, smell, and/or taste their artistic 
creations, they perceive the artistic process and the product, 
and they recall what they have learned. The experience of 
learning to read through the arts helps build their self- 
confidence and improve their self-image. 
STUDENT ELIGIBILITY 

Students are eligible for Chapter 1 services if they live in 
a targeted attendance area and score below a designated cut-off 
point on State-mandated standardized reading tests. The 
Chapter 1 Evaluation Reporting System specifies, however, that 
eligible students may aliso be selected for Chapter 1 programs on 
the basis of classroom performance. Generally, n^st children 
classified as limited English proficient (LEP) and many special 
education students are excluded from the standardized testing 
program and were selected for program participation on the basis 
of teacher ratings or classroom performances. 

STUDENTS SERVED 

During the 1988-89 school year, the program served lOO 
ungraded special education students and 716 students in grades 
two through six; 83 students were LEP students, and 633 were in 
general education classes. 



• More than three-qua'-ters of the students (78 percent) were 
in the general education curriculum. 

• The vast majority of students (81 percent) were in grades 
three, four, and five (see Table 1) . 

PROGRAM OBJECTIVES 

The objejtives for the 1988-89 L.T.R.T.A. program were: 

• General education students were expected to make 
statistically significant mean gains on standardized reading 
tests. 

• sixty percent of the LEP students were expected to master at 
least five additional reading skills on the posttest of the 
Wisconsin Design Skills Development Test that they had not 
mastered on the pretest. 

• Sixty percent of the special education students were 
expected to master at least three additional read ' skills 
on the posttest of the Wisconsin Design Skills Development 
Test that they had not mastered on the pretest. 

• All students were expected to show statistically significant 
mean gains from pretest to posttest on holistically scored 
writing samples. 

PROGRAM EVALUATION 

The purpose of the 1988-89 evaluation by the Of.Hce of 

Research, Evaluation, and Assessment/Instructional Support 

Evaluation Unit (OREA/I.S.E.U. ) was to describe the program and 

to assess its impact on student achievement in reading and 

writing. The following methods were used: 

• review of program documents and interviews of program 
staff to describe the program organization and funding, 
the curriculum, and staff development activities; 

• analyses of data retrieva) forms that report 
information about grade placement and students* 
attendance ; 

• observations of teacher/student interaction during 
instruction; and 
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TABLE 1 

Number and Percentage of Students by Grade Participating in 
the Learning to Read Through the Arts Program, 1988-89 





Number 
of Students 


Percentage 

of Stiudents 


2 


29 


3 . 6 


J 




1 / < u 


4 


296 


36.3 


5 


225 


27.3 


6 


27 


3.3 


Ungraded" 


100 


12.2 


Total 


816 


100.0 



Special education students are not placed by grade. 

• More than one-third of all students were in the fourth 
grade . 

• The vast majority of the students, 81 percent, were in 
grades three, four, and five. 

• Special education students constituted approximately 
one-eighth of the student population. 



• analyses of students' scores on standardized and 
criterion-referenced reading tests and holistically 
scored writing tests. 

SCOPE OF THE REPORT 

The purpose of this report is to assess the implementation 

and effectiveness of the 1988-89 Chapter 1 Learning to Read 

Through the Arts Program. Program organization and 

implementation are described in Chapter li. student attendance 

and academic achievement are discussed in chapter III. 

Conclusions and recommendations are offered in chapter IV. 
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II. PROGRAM ORGANIZATION AND IMPLEMENTATION 
PROGRAM BAC KGROUND AND FUNDING 

The Learning to Read Through the Arts Program (L.T.R.T.A.) 
operates under the auspices of the Office of Professional 
Development and Leadership Training under the Deputy Chancellor 
for Instruction and Development of the New York City Board of 
Education. The program has been in operation since 1971 and has 
received recognition as an exemplary program at the national, 
state, and local levels. Ir. 1988-89, the program was funded 
through Chapter 1 of the Education Consolidation and Improvement 
Act (E.C.I. A.) at $997,000. 
PROGRAM ORG A^.■ZATION 

The program was offered for a 3 2 -week cycle from October 
1988 to June 1989. Program activitif.'is were based at four sites — 
one each in Manhattan, the Bronx, Brooklyn, and Queens. In 
addition, field trips and workshops were conducted in association 
with the Staten Island Children's Museum, the Metropolitan Museum 
of Art, the Whitney Museum, the Guggenheim Museum, the Brooklyn 
Academy of Music, the National Society of Colonial Dames, the 
International Center of Photography, and the Staten Island 
Advance . 

At each program site, there were two reading teachers. At 
the Manhattan site, there were three artist teachers, and at the 
Bronx, Queens, and Brooklyn sites, there were two artist teachers 
and one office aide. At each site, one of the teachers acted as 
site coordinator. The program director, two staff developers, 
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and one office aide worked out of the program office, housed at 
P.S. 9 in Manhattan. 

Reading teachers are licensed, and several of them have many 
years with the program. All reading teachers are annualized — 
that is, on official Board of Education payrolls as regular 
teachers. Artist teachers are also licensed, but they are less 
experienced. However, with the exception of one teacher, they 
are not annualized. 

L.T.R.T.A. reading and artist teachers and participating 
classroom teachers work as a team to improve students' reading 
ard writing skills and to increase their interest in the arts. 
Reading teachers provide an individualized, diagnostic, 
prescriptive approach to reading. Artist teachers stress 
listening, speaking, reading, and writing in the arts-oriented 
workshops. Classroom teachers were expected to teach skills 
associated with the program one period each day at the students 
home school. 

When children visit the site, they are usually divided into 
two groups. One group receives an art lesson while the other 
group receives a reading lesson. The children switch classes at 
the end of tb lesson. This arrangement encourages the team- 
teaching concy..jt, where a reading teauh^^r and an artist teacher 
focus on the same or a si?nilar theme and coordinate their lesson 
plans. 

Most general education and LEP students were bused to the 
program site in their borough two afternoons a week for one four- 
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hour session. For the school year, they attended a total of 48 
sessions. In contrast , students from one school and special 
education students attended sessions one full day each week, for 
a total of 29 sessions. 

PROGRAM IMPLF;M7j!NTATI0N 
Site Observations 

An OREA/I.S.E.U. evaluator visited each site in March and 
May. During each visit, four classes were observed. Interviews 
were conducted with reading teachers, artist teachers, and leeder 
school classroom teachers who had accompanied their students to 
the sites on the day of the visit. 

The program buildings were clean and well maintained. 
Classrooms were well-illuminated, adequately heated and 
ventilated, and equipped with shelves, storage lockers, and 
moveable furniture. They were decorated with vocabulary lists, 
art terms, synonym-antonym charts, student tests and 
compositions, and teacher-made materials. Student art work was 
displayed in classrooms and throughout the school. It included 
self-portraits, paintings, murals, printing blocks, blueprints, 
tinfoil models, Japanese *ifish kites,** pottery, and painted 
screens . 
Curriculum 

All students attended reading and reading-oriented arts 
workshops. Special education and LEP students were taught the 
same curriculum as general education students, but at a slower 
pace. 
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As part of each reading workshop, teachers used creative 
visual materials that related to the site's general arts theme. 
Individual and small-group reading workshops focused on compre- 
hension, reasoning, problem-solving, the appreciation of 
literature, and reading for information and pleasure. Language 
skills instruction included word-identification skills, special- 
ized vocabulary used on norm-referenced tests, and writing. 

The reading-oriented art curriculum is based on a few themes 
each year. This year's themes included units on newspapers/ 
journalism and buildings in New York City. In the reading- 
oriented arts workshops, students' artistic experience included 
painting, print making, photography, sculpture, mixed media, 
theater, dance, and music. 

Reading and art are integrated as the workshop teacher leads 
a discussion of the project, the artistic process, or the 
students' creative experience. The teacher summarizes the 
discussion in a composition written in a master journal. As she 
writes, she describes important aspects of reading and writing — 
sounds and symbols, capitalization, punctuation, phrasing, and 
vocabulary. Students record the composition in their personal 
journals and then record their own experiences. 

In both reading and reading-oriented art workshops, the 
staff moved quickly into the lessons, clearly explained the main 
concepts, and related tasks to the objectives of the lesson. 
Classroom teachers assisted reading teachers, instructed students 
in word-recognition skills, and participated in the art 



8 



ERIC 



18 



workshops, students were engaged in their tasks and worked 
independently and efficiently. 
STAFF DEVELOPMENT 

The major objectives of staff development were to share 
educational information and to learn to implement the program 
successfully. The project director, staff developers, and site 
coordinators provided training for staff teachers and classroom 
teachers . 

In the fall, before the start of the program, staff members 
and classroom teachers attended pre-service training sessions. 
In addition, artist teachers, reading teachers, and classroom 
teachers attended in-service training sessions during the school 
year. Staff development topics included the interpretation of 
standardized testing and diagnostic inventories, the 
developmental breakdowns of reading, language, and writing 
skills, and strategies for developing thinking skills. 

Staff participated in seven staff development sessions each 
month. Once a week, the site coordinator at each site conducted 
a staff development session. Once a month, a staff developer 
visited each site, the program director conducted a workshop at 
each site, and the staff from each site attended a staff 
development meeting at the program office in Manhattan. 

STAFF PERCEPTIONS OF THE PROGRAM 

Teachers were asked for the two most positive aspects of the 
program. The most frequent responses were that the integration 
of reading and art helped children to read better, the program 
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helped students become more motivated to learn, and it improved 
their attitudes towards learning. 

In general, teachers felt the program was well organized and 
operated smoothly. They praised the cooperative attitude of the 
administrative staff, team leaders, and feeder school personnel. 
In addition, they singled out the excellent rapport among 
children and between children and program teachers. Finally, 
teachers thought highly of staff development training and were 
satisfied with the quality and types of program materials. 

Teachers believed in the program philosophy. They felt that 
the organization necessary to complete an art work instilled 
discipline in students, the careful planning required in the 
production of art developed students' ability to identify cause 
and effect relationships and the sequencing of items, the 
attention to detail and clarity of thought demanded in art were 
helpful in writing compositions, and the expansion of students' 
scope of interests widened children's language experience base. 
Teachers also thought that art itself was beneficial for children 
because children's self-confidence increases as they create 
something in the art workshop and because art is an outlet for 
creative children and helps many children open up to their peers. 

Finally, teachers noted two problems with the program. 
First, there were occasional problems with the timely arrivals of 
buses. More importantly, however, they felt that the 
professional status of artist teachers was a problem. Because 
artist teachers are not annualized, they leave the program when 
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economic demands increase or the need for fringe benefits become 
paramount. Teachers expressed the conviction that regularizing 
the art positions would improve the progi^am by increasing its 
stability and continuity. 
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III. STUDENT OUTCOMES 

ATTENDANCE 

The maximum number of days of attendance in the Learning to 
Read Through the Arts Program was 48 or 29 days, depending on the 
type of student involved. Most general education and LEP 
students could attend workshops two afternoons a week for a 
maximum of 48 four-hour sessions. General education and LEP 
students from one school and special education students could 
attend workshops one full day each week for a maximum of 29 
sessions. Overall, program students attended 83 percent of the 
sessions. The mean number of days absent was 7.1 days. 
METHODS USED TO EVALUATE STUD ENT ACHIEVEMENT 

The impact of the program on student achievement in reading 
and writing was determined by evaluating students' performance on 
standardized and norm-referenced reading tests and holistically 
scored writing tests against the program objectives. The program 
objective for general education students on standardized reading 
tests— the Metropolitan Achievement Test (MAT) and the Degrees of 
Reading F wer Test (D.R.P.)— was that they would achieve a 
statistically significant mean gain from pretest to posttest. On 
the Wisconsin Design Skills Development Test, LEP students were 
expected to master five additional reading skills and special 
education students were expected to master three additional 
reading skills on the posttest that they had not mastered on the 
pretest. On the holistically scored writing tests, students were 
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to achieve statistically significant mean gains from pretest to 
posttest. 

Reagiy ^q Tests of General Education Studi ^nty 

On the standardized reading tests, students* raw scores were 
organized by grade and converted to normal curve equivalents 
(N.C.E.s).* students in grade three took the MAT in spring 1988 
and the D.R.P. test in spring 1989. Scores on the D.R.p. 
(posttest) were converted to (pretest) MAT scores so that 
achievement could be measured.** However, since the MAT 
emphasizes basic skills and the D.R.P. emphasizes reading 
comprehension, the comparison may not be a good measure of 
student achievement. 

Statistical analyses were carried out on the converted 
N.C.E. scores. Correlated t-tests were used to determine whether 
mean gains were statistically significant. Statistical 
significance indicates whether the changes in achievement are 
real or occur by chance. However, achieving statistically 
significant mean gains does not address the issue of whether the 
mean gains are important to the students' educational 



♦Normal curve equivalent scores are similar to percentile 
ranks but, unlike percentile ranks, are based on an equal- 
interval scale ranging from 1 to 99, with a mean of 50 and a 
standard deviation of approximately of 21. Because N.C.E. scores 
are equally spaced, mathematical and statistical calculations 
such as averages are meaningful; in addition, comparisons of 
N.C.E. scores may be made across different achievement tests. 

** The cqui-percentile method was used. The frequency 
distributions of scores on the tests were employed to generate 
conversion tables, which were approved by the State Education 
Department. 
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development. Tor example, the importance of achieving 
statistically significant mean gains can be exaggerated for large 
groups of students because even small mean gains by large groups 
of students will generally be statistically significant. 
Similarly, the importance of not achieving statistically 
significant mean gains can be overstated for small groups of 
students because it is more difficult for small groups to achieve 
mean gains that are statistically significant. Thus, an effect 
size (E.S.)* is reported for each mean difference to indicate 
whether each mean gain or loss was educationally meaningful. 
Reading Tests of LEP and Sp ecial Education Students 

On the Wisconsin Design Skills Development Test, tallies 
were made of the number of reading skills mastered by each 
student on the pretest and posttest, and the number of additional 
skills mastered by each student on the posttest was calculated. 
Frequency distributions were performed co identify the number of 
special education students and the number of LEP students by 
grade and overall who mastered a specific, additional number of 
skills. 



*The effect size, developed by Jacob Cohen, is the ratio of 
the mean gain to the standard deviation of the gain. This ratio 
provides an index of improvement irrespective of ,the size of the 
sample. According to Cohen, .2 is a small effect size, .5 is a 
moderate effect size, and .8 is a large effect size. Only effect 
sizes of .8 and above are considered educationally meaningful. 
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Writhing Tc^st^s 

Writing tests were organized by grade and holistically 
scored using a five point scale that measures how well a student 
jommunicates . Test readers from the program staff were asked to 
pay special attention to the coherence of the passage, its 
sentence structure and sequence, and the writer's awareness of 
the audience. Each writing test was independently scored by two 
readers. A one point discrepancy between readers was acceptable. 
However, discrepancies of more than one point required a third 
reader. The program reported that 98 percent of the cases did 
not need a third reader. 

After the tests had been scored, correl.ited t-tests were 
used to determine statistical significance. Then, an effect size 
was calculated for each mean difference to indicate whether each 
mean gain or loss was educationally meaningful. 

AC ADEMIC ACH IEVEMENT IN READING 
General Education Student s 

Table 2 presents data on second and third grade general 
education students' achievement on standardized reading tests. 
Second grade students were pretested in spring 1988 and 
posttested in spring 1989 on the MAT. In contrast, third grade 
students were pretested in spring 1988 on the MAT and posttested 
in spring 1989 on the D.R.P. D.R.P. scores were converted for 
comparison with MAT scores. Mean differences were calculated and 
measured against the program objective, a statistically 
significant mean gain. Table 2 shows that; 
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Mean N.C.E, Differences of General Education Students by Grade 
in the Learning to Read Through the Arts Program 
on the Metropolitan Achievement Test, 1988-89 







Pretest 


Posttest 


Difference 


Effect 


Grade 


N 


Mean 


S.D. 


Mean 


S.D. 


Mean S.D. 


Size 


2 


20 


35.1 


13.2 


34.3 


11.3 


-0.8 14.3 


0.1 


3' 


69 


32.3 


13.0 


36.7 


16.9 


4.4'' 17.5 


0.3 



" Third grade students were pretested on the MAT in spring 1938 
and posttested on the Degrees of Reading Power Test (D.R.P.) in 
spring 1989. The new (1987-88) D.R.P. norms were used on spring 
1989 test scores. Then, D.R.P. scores were converted for 
comparison with MAT scores. 

^ The mean difference was statistically s.^^nificant at the p<.05 
level . 

• Third grade students achieved a s<-atistically 
significant mean gain of 4.4 N.C.L.s. This mean gain 
represented a small effect size. 

• Mean N.C.E. scores for second grade students decreased 
from pretest to posttest, from 35.1 N.C.E.s to 34.3 
N.C.E.s. However, the decrease of 0.8 N.C.E.s was not 
statistically significant. 
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• Third grade students achieved a statistically 
significant mean gain of 4.4 N.C.E.s. This mean gain 
represented a small effect size. 

• Mean N.C.E. scores for second grade students decreased 
from pretest to posttest, from 35.1 K.C.E.s to 34.3 
N.C.E.s. However, the decrease of 0.8 N.C.E.s was not 
statistically significant. 

Table 3 presents data on student achievement by general 
education students in fourth, fifth, and sixth grade on the 
D.R.P.. Students were pretested in spring 1988 and posttested in 
spring 1989. The new (1987-88) D.R.P. norms were used on the 
spring 1988 and spring 1989 test scores. Kean differences were 
calculated and measured against the program objective, a 
statistically significant mean gain. Table 3 shows that: 

• The overall mean gain of 9.9 N.C.E.s was statistically 
significant. The overall effect size represented an 
educationally meaningful mean gain. 

• Students in grades four, five, and six achieved 
statistically significant mean gains. Effect sizes for 
student!? in fifth and sixth grade represented 
educationally meaningful gains. 

i:,iinj.te(^ E nglish Prpficient gtM<a<gnts 

Table 4 presents data on student achievement in reading by 
LEP students in third, fourth, and fifth grade on the criterion- 
referenced Wisconsin Design Skills Development Test. Students 
were pretested in fall 1988 and posttested in spring 1989* 
Student achievement was measured against the program objective 
that 60 percent of LEP students would master at least five 
additional reading skills on the posttest that they had not 
mastered on the pretest. Table 4 shows that: 
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TABLE 3 



Mean N.c.E. Differences of General Education students by Grade 
in the Learning to Read Through the Arts Program 
on the Degrees of Reading Power Test, 1988-89* 



Grade 


N 


Pretest 


Posttest 


Difference ^ 


Effect 
Size 


Mean 


S.D. 


Mean 


S.D. 


Mean 


S.D. 


4 


192 


29.7 


13.1 


35.7 


15.3 


6.0 


13.4 


0.4 


5 


175 


23.5 


13.4 


37.4 


13.4 


13.9 


11.3 


1.2 


6 


22 


24.9 


10.4 


37.5 


11.6 


12.6 


11.1 


1.1 


Total 


389 


26.6 


13.4 


36.5 


14.3 


9.9 


12.9 


0.8 



• The new (1987-88) D.R.P. norms were used on the spring 1988 and 
spring 1989 test scores. 

*• Mean differences WGre statistically significant at the p<.05 
level. 



• The overall mean gain of 9.9 N.C.E.s was statistically 
significant. The overall effect size represented an 
educationally meaningful mean gain. 

• Students in grades four, five, and six achieved 
statistically significant mean gains. Effect sizes for 
students in fifth and sixth grade represented 
educationally meaningful gains. 
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TABLE 4 



Number and Percentage of Limited English Proficient Students 
by Grade in the Learning to Read Through the Arts Program 

Mastering Five or More Additional Skills 
on the Wisconsin Design Skills Development Test, 1988-89' 



Grade 



Number of 
Students 



Additional Reading Skills M astered 
Less than Five More than Five 

JSL -JL. JL. 



3 


39 


7 


18 


32 


82 


4 


36 


8 


22 


28 


78 


5 


7 


0 


0 


7 


100 


Total 


82 


15 


18 


67 


82 



' The program objective was that 60 percenc of LEP students would 
master at least five additional reading skills on the posttest 
that they had not mastered on the pretest. 

• More than 60 percent of LEP students in each grade and 
82 percent of all LEP students mastered five or more 
additional reading skills on the posttest. 

• All seven fifth grade LEP students mastered five or 
nore additional reading skills on the vosttest. 
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• More than 60 percent of LEP students in each grade and 
82 percent of all LEP students mastered five or more 
additional reading skills on the posttest. 

Table 5 presents data on student achievement in reading by 
special education students on the criterion-referenced Wisconsin 
Design Skills Development Test. Students were pretested in fall 
1988 and posttested in spring 1989. Student achievement was 
measured against the program objective that 60 percent of the 
special education students would master at least three additional 
reading skills on the posttest that they had not mastered on the 
pretest. Table 5 shows that: 

• Eighty-one percent of special education students 
mastered three or more additional reading skills on the 
posttest . 

Comparison of Reading Achiev ement Over Four School Years 

Table 6 presents data on student achievement by general 
education students on the D.R.P. over the last four school years, 
1986-87 through 1988-89. Students were pretested in the spring 
of the previous year and posttested in the spring of the current 
academic year. For example, in the 1988-89 school year, students 
were pretested in spring 1988 and posttested in spring 1989. The 
new (1987-88) D.R.P. norms were used on the pretest (spring 1988) 
and posttest (spring 1989) test scores. Mean differences were 
calculated and measured against the program objective, a 
statistically significant mean gain. Table 6 shows that: 

• Mean N.C.E. gains each year were statistically 
significant. 
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TABLE 5 



Number and Percentage of Special Education students 
in the Learning to Read Through the Arts Program 
Mastering Three or More Additional Reading Skills 
on the Wisconsin Design Skills Development Test, 1988-89* 



Number of Additional 
Skills Mastered 



Number 
of Students 



Percentage 
of Students 



Less than three 
Three or more 



Total 



19 
81 
100 



19 
81 
100% 



The program objective goal was that 60 percent of special 
education students would master at least three additional reading 
skills on the posttest that they had not mastered on the pretest. 

• Eighty-one percent of special education students 
mastered three of more additional reading skills on the 
posttest . 



21 



ERIC 



32 



TABLE 6 



Mean N.C.E. Differences of General Education Students 
in the Learning to Read Through the Arts Program 
on the Degrees of Reading Power Test, 1986-87 through 1988-89 



Year 


Grade 


Number of 
Students 


Mean N.C.E. 
Differences* 


1986-87 


4-5 


343 


13.1 


1987-88 


4-5 


303 


4,2 


1987-88 


4-6 


347 


3.9 


1988-89'* 


4-6 


389 


9.9 



' Mean gains were statistically significant at the p<.05 level. 



^ The new (1987-88) D.xx.P. norms were used on the pretest (spring 
1988) and posttest (spring 1989) test scores. 

• Mean N.C.E. gains each year were statistically 
significant. 

• The mean gain of 9.9 N.C.E.s in 1988-89 is 
substantially larger than gains in the previous two 
school years. 
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• The mean gain of 9.9 N.C.E.s in 1988-89 is 
substantially larger than gains in the previous two 
school years. 

ACADFMIC ACHIEVEMENT IH WRITING 
General Education Students 

Table 7 presents data on student achievement by general 
education students in grades two through six on holistically 
scored writing tests. Students were pretested in fall 1988 and 
posttested in spring 1989. Mean differences were calculated and 
measured against the program objective, a statistically 
significant mean gain. Table 7 shows that: 

• The overall mean gain of 1.8 raw-score points was 
statistically significant. The overall effect size 
represented an educationally meaningful gain. 

• Students in all grades achieved statistically 
significant mean gains. Effect sizes for students in 
all grades represented educationally meaningful gains. 

Limited English Proficient Students 

Table 8 presents data on student achievement by LEP students 

in grades three through five on holistically scored writing 

tests. Students were pretested in fall 1988 and posttested in 

spring 1989. Mean differences were calculated and measured 

against the program objective, a statistically significant mean 

gain. Table 8 shows that: 

• The overall mean gain of 2.0 raw-score points was 
statistically significant. The overall effect size 
represented an educationally neaningful mean gain. 

• Students in all grades achieved statistically 
significant mean gains. Effect sizes for students in 
all grades represented educationally meanin^fful gains. 
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TABLE 7 



Mean Raw-Score Differences Of General Education Students 
by Grade in the Learning to Read Through the Arts Program 
on Holistically Scored Writing Tests, 1988-89 







Pretest 


Posttest 


Difference* 


Effect 




N 


Mean 


S.D. 


Mean 


S.D. 


Mean 


S.D. 




2 


18 


1.1 


1.0 


4.0 


1.3 


2.9 


1.3 


2.2 


3 


90 


1.7 


0.9 


3.6 


1.2 


1.9 


1.3 


1.5 


4 


215 


2.1 


1.0 


4.0 


1.1 


1.9 


1.1 


1.7 


5 


187 


2.7 


1.0 


4.3 


1.2 


1.6 


1.1 


1.5 


6 


22 


3.9 


0.6 


5.1 


0.8 


1.2 


0.9 


1.3 


Total 


532 


2.3 


1.0 


4.1 


1.2 


1.8 


1.2 


1.5 



* Mean differences were statistically significant at the p<.05 
level . 



• The overall mean gain of 1.8 raw-score points was 
statistically significant. The overall effect size 
represented an educationally meaningful gain. 

• Students in all grades achieved statistically 
significant mean gains. Effect sizes for students in 
all grades represented educationally meaningful gains. 



24 



TABLE 8 



Mean Raw Score Differences of Limited English Proficient Students 
by Grade in the Learning to Read Through the Arts Program 
on Holistically Scored Writing Tests, 1988-89 



Grade 


N 


Pretest 


Posttest 


Difference • 


Effect 
Size 


Mean 


S.D. 


Mean 


S.D. 


Mean 


S.D. 


3 


34 


1.5 


1.1 


3.2 


1.2 


1.7 


0.9 


1.9 


4 


29 


1.0 


1.1 


3.1 


1.9 


2.1 


1.5 


1.4 


5 


6. 


0.8 


1.3 


4.0 


0.6 


3.2 


1.5 


NA 


Total 


•69 


1.2 


1.2 


3.2 


1.5 


2.0 


1.3 


1.5 



" Mean differences were statistically significant at the p<.05 
level. 



• The overall mean gain of 2.0 raw-score points was 
statistically significant. The overall effect size 
represented an educationally meaningful mean gain. 

• Students in all grades achieved statistically 
significant mean gains. Effect sizes for students in 
all grades represented educationally meaningful gains. 
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Special Education Students 

Table 9 presents data on student achievement by special 
education students on holistically scored writing tests. 
Students wure pretested in fall 1988 and posttested in spring 
1989. Mean differences were calculated and measured against the 
program objective, a statistically significant mean gain. Table 
9 shows that: 

• The statistically significant mean raw-score gain of 
1.9 points met the program criterion for success. It 
represented an educationally meaningful gain. 
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TABLE 9 



Mean Raw Score Differences of Special Education Students 
in the Learning to Read Through the Arts Program 
on Holistically Scored Writing Tests, 1988-89 







Pretest 


Posttest Difference ' 


Effect 




Grade N 


Mean S.D. 


Mean S.D. Mean S.D. 


Size 




Ungraded** 84 


1.7 1.4 


3.6 1.1 1.9 1.1 


1.7 



' The mean difference was statistically significant at the p<.05 
level. 



Special education students are not placed by grade. 

• The statistically significant mean raw-score gain of 
1.9 points met the program criterion for success, a 
statistically significant mean gain. It represented an 
educationally meaningful gain. 
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IV. CONCLUSIONS AND RECOMMENDATIONS 

CONCLUSIONS 

The evaluation of student achievement on reading and writing 
tests shows that with one exception — the performance of second 
grade students in the general education curriculum on the 
Metropolitan Achievement Test (MAT)— all program objectives for 
student improvement in reading and writing were met or surpassed. 
However, it must be noted that a significant number of students 
did not take both the pretest and the posttest in reading and 
writing: namely, 660 students (81 percent) took both tests in 
reading and 689 students (84 percent) took both tests in writing. 

General education students in all grades except grade two 
made statistically significant mean gains from pretest to 
posttest on standardized reading tests, meeting the program 
objective. For students in fifth and sixth grade, effect sizes 
represented educationally meaningful gains. Moreover, for 
students in third, fourth, fifth, and sixth grades, all mean 
pretest scores were below the State Education Department's 35 
N.C.E. threshold for educationally disadvantaged students, while 
all mean posttest scores were above it. In conjunction with the 
achievement of statistically significant mean gains, this 
indicates the positive impact of the program on students' ability 
to read. 

Second grade students in the general education curriculum 
did nffit meet the program objective in reading achievement. In 
fact, mean scores dropped from 35.1 N.C.E.s to 34.3 N.C.E.s from 
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pretest to posttc<i»t. However, this small decrease was neither 
statistically sicpiif icant nor educationally meaningful. 
Moreover, only 2G of 29 second grade students in the general 
education curriculum (69 percent) took both pretests and 
posttests. The percentage was the lowest for any grade or 
population in the program. Therefore, the low number of students 
(15 is the minimum number of cases for which statistical 
significance should be validly tested) , the low percentage, and 
the fact that the mean decrease was not statistically significant 
suggest that the impact of the program on the reading achievement 
of second grade students this year was not adequately measured by 
the standardized reading test. 

On the Wisconsin Design Skills Development Test, 81 percent 
of special education students mastered three or more additional 
reading skills on the posttest that they had not mastered on the 
pretest, and 82 percent of LEP students mastered five or more 
additional reading skills. Thus, the program surpassed its 
objectives for reading achievement for special education and LEP 
students. In addition, the breakdown of LEP student reading 
achievement by grade showed that the program surpassed its 
objective for each grade. 

On the holistically scored writing tests, special education 
students and LEP and general education students in all grades 
made statistically significant mean gains and thus met the 
program objectives. These gains represented large effect sizes 
and were educationally meaningful. 
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The review of program documents, observation of program 
activities, and interviews of program staff indicated that the 
program wa& implemented as proposed* The program was well 
organized and effective. Program staff were professional and 
collegial. Students were engaged and enthusiastic. However, 
there were some small problems. At some sites, there were 
occasional problems with the timely arrival of buses. In 
addition, staff interviews brought out the fact that the non- 
annualized status of artist teachers was a source f anxiety. 
This status was perceived by teachers to contribute to a high 
turnover among artist teachers and thus less stability and 
continuity in the program. 
RECOMMENDATIONS 

The program has been implemented as proposed, and students 
in general have met the program objectives for achievement in 
reading and writing. 

• The program should continue its innovative curriculum 
of integrating reading and the arts and, as several 
teachers requested in the interviews, expand, if 
possible, to serve an even larger population of 
students . 

Students travel from their home school to the program sites 
and on field trips to cultural institutions in New York City. 
Staff interviews indicated that transportation is an occasional 
problem at some sites. While transportation problems are a fact 
of life in New York City, the timely transportation of students 
is central to the implementation of the program. 

• Given its importance to the program, transportation 
should be closely monitored by program staff so that 
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any decline in service can be quickly identified and 
corrected . 

Staff interviews also revealed that, with the exception of 

one artist teacher, artist teachers are not annualized. This 

form of job insecurity is a source of anxiety to artist teachers. 

Program teachers believe it leads, to a higher turnover rate among 

artist teachers than among reading teachers and thus to less 

program stability and continuity, both of which are essential to 

the team teaching concept practiced in the program. 

• The evaluation of the 1989-90 progrem should 
investigate whether or not the perceived higher 
turnover rate of artist teachers is a fact. If the 
turnover rate is substantially higher, this would be 
evidence of a lack of program stability and continuity. 
OREA, program staff, other Board of Education staff, 
and/or interested parties might then develop a proposal 
to change the status of artist teachers in a feasible 
manner. 



